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[57] ABSTRACT 

A plurality of raster-scanned image variables are time- 
compressed and time-division multiplexed on a line- 
sequential basis for pyramid analysis. The resulting 
differential pulse code modulation is statistically en- 
coded and assembled together with digital audio code 
and synchronizing code. The assembler supplies a serial 
stream of bits to a transmitter. Receiving apparatus for 
a video teleconfi^encing station includes a receiver 
complementary to the transmitter and a parser for sepa- 
rating digital codewords. Those codewords that are 
statistical codes descriptive of differential pulse code 
modulation of video are demodulated. The demodula- 
tion results are formatted for a pyramid synthesizer. 
The stream of samples provided as output signal from 
the pyramid synthesizer is time-division demultiplexed 
and time expanded to recover the plurality of raster- 
scanned image variables. 

14 Claims, 5 Drawing Sheets 
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generate descriptions of the iniages. The streams of 

LINE-SEQUENTIAL PYRAMID PROCESSING OF digital data generated by the spatial-frequency analysis 

A PLURALITY OF RASTER-SCANNED IMAGE pirocedure can be condensed into a single stream of 

VARIABLES digital bits for coding purposes. This is generally a pref- 

5 erable procedure to do. This procedure is especially 

The present invention relates to the pyramid process- satisfactory when the coding procedure involves the 

ing of a plurality of raster-scanned image variables step of temporal differential pulse code modulation of 

being carried out substantially parallel in time. the type known as "delta-frame". In delta-frame DPCM 

nAne-opninsin -rm TMVTSKrrrrixi im^se samples separated one frame duration in time are 

BACKGROUND OF THE INVENTION differentiaUy combined and the resultant difference is 

One place parallel-in-time pyramid processing of a quantized to generate a differential pulse code modula- 
plurality of raster-scanned image variables is of interest tion (DPCM) signal. The procedure is especially satis- 
is in increased depth-of-focus cameras as described by factory because the distributions of the amplitudes of 
E. H. Adelson in U.S. Pat No. 4,661,986 issued 28 Apr. DPCM signal responses to different image variable 
1987 and en titled ''DEPTH-OF-FOCUS IMAGING 15 components usually tend to resemble each other more 
PROCESS METHOD". Another place such parallel- than do the distributions of the amplitudes of the com- 
in-time pyramid processing of a plurality of raster- ponents themselves. DPCM signal amplitudes tend to 
scanned image variables is of interest is in the splining have Laplacian distributions with zero means, 
together of component sub-images from respective A number of different pyramid processing proce- 
cameras to form as a mosaic a complete image too large 20 dures are known, and in the digital transmission of im- 
in area to be acquired by a single camera. Such a proce- ages it is generally desirable to carry any such proce- 
dure is taught by P. J. Burt in his U.S. patent application dure forward on a pipeline basis. In U.S. Pat No. 
No. 020,818 fUcd 2 Mar. 1987, cntitied "PYRAMID 4,674,125 issued 16 June 1987, entitled "REAL-TIME 
PROCESSOR FOR BUILDING LARGE-AREA, HIERARCHAL PYRAMID SIGNAL PROCESS- 
HIGH-RESOLUTION IMAGE BY PARTS" and 25 ING APPARATUS", and incorporated herein by ref- 
assigned at time of filing to RCA Corporation, merged erence, C. R. Carlson, J. H. Arbeiter and R. F. Bessler 
18 Dec. 1987 into General Electric Company. Another describe pipeline procedures for implementing pyramid 
place such parallel-m-time pyramid processing is of processing procedures, especially the Burt pyramid 
interest is in the description of full color images m terms processing procedure well known to those skilled in the 
of component color variables. These component color 30 art C, H, Anderson describes another pyramid process- 
signals may consist of luminance (Y), in-phasc chromi- mg procedure suitable for pipelined fanplemcntation in 
nance (I) and quadrature-phase chrominance (Q) van- U.S. Pat No. 4,718,104 issued 5 Jan. 1988 and entitied 
ables, by way of example or may consist of red (R), "FILTER-SUBTRACT-DECIMATE HIERAR- 
green (G) and blue (B) tri-stimulus variables, by way of CHAL PYRAMID SIGNAL ANALYZING AND 
further e3an^>le;*or may consist ofother groups of vari- 35 SYNTHESIZING TECHNIQUE," In the Burt 
ables, as known to those skilled in the art of colorime- pyramid and in the Anderson F-S-D pyramid, ail spa- 
try- tial-frequency analysb components are omnidirectional 

Pyramid processing includes spatial-frequency analy- in nature. Despite decimation by 2:1 in each spatial 

sis done by an iterative spatial filtering procedure per- direction from each octave to the next lower in the 

formed (usually in two orthogonal spatial dimensions) 40 spatial-fi-equency analysis by octaves, the total number 

on images as described in sampled-data terms, which of samples in the set of spatial-frequency analyses (in the 

processing dissects the image into component spatial- limit) increases by one-third. 

frequency spectra, the dissection process analyzing an Newer pyramid processing procedures separate each 

image variable into component band-pass octave-width octave band of spatial-frequency components into three 

spectra and a remnant low-pass spectrum. The band- 45 portions, one having similar value horizontal and verti- 

pass spectra are sometimes referred to as "Laplacian" cal components in the octave, one having only excess 

responses and the remnant low-pass spectrum, as a horizontal component in the octave, and one having 

"Gaussian" response. only excess vertical component in the octave. These 

Pyramid processing may also include a subsequent three portions can be spatially decimated, so the total 
synthesis of an image-descriptive variable from its com- 50 number of samples in the set of spatial analyses of a 
ponent spatial-frequency spectra. Since the successively signal is no larger than the number of regularly spaced 
lower-spatial-frequency octaves are customarily sam- samples in that original signal itself. Such pyramid pro- 
pled at successively lower density in each spatial- cessing procedures are described by J. W. Woods and S. 
dimension in the analysis procedure, these lower-spa- D. O'Neill in their paper "Subband Coding of Images" 
tial-frequency octaves and the renmant low-pass spec- 55 appearing on pp. 1278-1288 of the IEEE TRANS AC- 
trum are resampled using interpolativc techniques to TIONS ON ACOUSTICS. SPEECH, AND SIGNAL 
permit the additive combination of the various spectral PROCESSING, Vol ASSP-34, Oct. 1986; are described 
components at the fiiU sampling density of the image by S. Mallat in his paper "An Efficient Image Reprcsen- 
variable recovered by the subsequent synthesis proce- tation for Multiscale Analysis" in the TOPICAL 
dure. 60 MEETING ON MACHINE VISION summaries of 

Pyramid processing is useful, for example, in the papers presented at the Machine Vision Topical Meet- 
digital transmission of images, with the spatial-frc- ing 18-20 Mar. 1987 at Incline Village, Ncv., as pub- 
quency analysis being performed at the sending station lished by the Optical Society of America, Washington, 
to generate streams of digital data for subsequent cod- DC; and are described by A. H. Tan, K-M Liu, K-H 
ing to remove redundancy before transmission. The 65 Tzou and E. B. Vogel in their p^er "An Efficient 
transmission is decoded at the receiving station to re- Pyramid Image Coding System" in the PROCEED- 
generate the stream or streams of digital data which are UVfCi5 iCASaP 6 7, Vor 2, pp io.6. i-4, puuiA^ucu uy ilic 
used in a spatial-frequency synthesis procedure to re- IEEE. 
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The reiterative nature of the digital filtering in the variables are apt to be unalike in value* Tune-divi- 
pyramid processing and the successive spatial decima- sion multiplexing on a sequential-field or (sequential- 
tion by octave have led fellow workers in the field to frame) basis docs not tend to introduce more high-spa- 
consider whether hardware savings naay be effected by tial-frcquency content into the serial stream. However, 
time-division-multiplcxing of multiply-nsed digital 5 tends to require a burdensome amount of temporary 
hardware in pipelined pyraimd proc«sstn fy, compressing and interleaving the fields 

'L'^^^^^^' (or frames). That is, the processing of the pluraUty of 

FLEXED REAL-TIME PYRAMID SIGNAL PRO- v«ri«WA« i« Zm n«r«*H mit JLm 

CESSmO SYSTEM" Issued 24 Nov, 1987 to R. F Si^ciri^:"""" " 

Bessler, J. H. Arbeiter and J. O. Smnmger, m which 10 4-u « . ^ . ^ • 

multiplexinq of diqital hardware aUows the same hard- . P^^* inventor finds upreferaW^ to tmie-dm. 

ware to be used in a succession of the successive spatial- «on-muHipl« the plurahty of unage vanables on hnc- 

frequency filtering procedures used in developing sepa- sequentud basis m the input signal to a pyramid analyzer 

rate octaves of filter response from a single signal. ^r m the output signal from a pyramid synthesizer. The 

Of interest is U.S. Pat. No. 4,447,886 issued 8 May 15 raster-scanned composite image field resulting from 

1984 to G. William Meeker and entitled 'TRIANGLE ^^^^ multiplexing resembles the component image 

AlSD PYRAMID SIGNAL TRANSFORMS AND fields of the respective image variables arranged in 

APPARATUS". In reference to FIG. 38 of his patent sidc-by-side relationship. Not much additional high-spa- 

Meeker describes time-division mutiplexed use of tial-frequency content is introduced by line-sequential 

pyramid analyzer apparatus sequentially selecting Y, I 20 time-division multiplexing, and the temporary storage 

and Q samples to each processing block transformer in requirements for such multiplexing are not onerous, 
every successive band of spatial-frequency analysis. 

This differs from time^Uvision multiplexing the mput SUMMARY OF THE INVENTION 
signal to a complete pipeline processing system, as the A first aspect of the present invention is the line- 
present inventor advocates, inasmuch as the input signal 25 sequential spatial-frequency fateiing of a plurality of 
to each successive stage of pyramid analysis must be different raster-scanned variables, each having the same 
separately time-division-multiplexed witii regard to its number of scan lines per frame. The variables are sup- 
input signal. The Meeker procedure presents cumber- pii^^ to respective time compressors each having ca- 
some control problems m dedicated digital hardware ^ of samples. TTie scan 
used for pyramid analyse, A si^^ can be 30 compressors resample the raster-scamied variables 

made with regard to dedicated digital hardware used *;«.J4;,^.;«„ ™„if;^-* n,-^ « ^. f i 

for pyramid synthesis that uses a time-division-multi- ^ .timc-di>^ion muluplex thenj on a hne-sequcntial 

plexing procedure similar to that which Meeker advo- effect the hne-sequential samp^g of these 

cates for a pyramid analyzer. The Meeker procedure ^^P^^^e vanables arranges their sample fields side by 

and variants of it are even more awkward to implement 35 ^'^^ ^ components of a larger, raster-scanned sample 

in pyramid processing hardware when I and Q are sub- resultant compoate signal raster-scanning the 

sampled respective to Y. \9i$^t sample field is applied as input signal to a subse- 

A constraint that limited prior art thinking is the fluent spatial-frequency filter, 

consideration of the color variables simply in terms of further aspect of the invention the spatial filter is 

separate respective image fields each conformally map- 40 & pyramid analyzer In a still further aspect of the inven- 

ping point-by-point to the original scene presented to tion the pyramid analyzer results are reformatted and 

the camera according to the same mapping transform. subjected to differential pulse code modulation and 

The present inventor removed this constraint upon statistical coding procedures. 

thinking about pyramidal processing of data, with the In yet another aspect of the invention, the spectral 
realization that a plurality of image-descriptive van- 45 analyses from such a line-sequential pyramid analyzer, 
ables can be time-division multiplexed into a single se- or the lower-spatial frequency components thereof, are 
rial data stream for plural-stage pipelined pyramid pro- subsequenUy (e.g. after transmission through a data 
cessing, which serial data stream describes a raster- li^k) supplied to a pyramid synthesizer, and the synthc- 
«:anned composite image field. This composite image s^ppu^d to time^ivision-demultiplexing 
field does not conformaUy map pomt-by-pomt to the 50 time-decompressing apparatus to recover repUcas 
ongmal scene presented to the c^em. Ratiier plural of different raster-scamied variables, 
pomts m the composite image field, one for each van- 
able, conformally map to the original scene presented to BRIEF DESCRIPTION OF THE DRAWING 
the camera. Multi-stage pipelined pyramid processing is , ^ 

perfonned on an inpuTsi^ generated by timcKUvision 55 ^ * "f^""^ ^« *^PP^' 

multiplexing the sLp\^ of a plundity of image vari- ^ ^ video tdeconferencmg stauon. which sendmg 

ables together in a serial data stream. apparatus embodies an aspect of the mvention. 

Time^division multiplexing is preferably done so as to ^ " * schematic diagram of a scan line compres- 

avoid the introduction of excessive high-spatial-fi^ sor as can be used m the FIG. 1 apparatus, 

quency content into the serial stream describing the 60 ^^^S. 3 and 4 are drawmgs useful in explaining the 

raster-scanned composite image field. This facilitates composite unage fields used in line-sequential pyramid 

dropping the transmission of high-spatial-frequency processing a plurality of raster-scanned image variables 

content at times the bandwidth capabilities of the data according to the principles of the invention, 

link tena otiierwisc io uc cxc^icd, -without cccirrrlng FIO- 5 is a schematic diagram of the receiving appa- 

unacceptable loss in the quality of the transmitted color 65 ratus in a video teleconferencing station, which receiv- 

image. Time-division multiplexing a plurality of vari- ing apparatus embodies an aspect of the invention, 

ables on a sequential-pixel basis strongly tends to intro- FIG. 6 is a schematic diagram of a scan line decom- 

duce high-spatial-frequency content since at most times pressor as can be used in the FIG. 5 apparatus. 
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DETAILED DESCRIPTION assembly process be one that is designed to accommo- 
date the fact that uniform duration segments of digital 

FIG. 1 illustrates the use of line-sequential pyramid video and digital audio are coded to form respective bit 

processing in the sending apparatus of a video tclecon- stream segments that vary in duration. 

fercncing station. A color television camera 2 is of a 5 Control circuitry 11 synchronizes operation of the 

type supplying three color-descriptive digital signals various parts of the FIG. 1 sendmg apparatus. One 

(eg. Y,I,Q or R,G,B) paraUel in time to a time^vision aspect of this synchronization of operation, already 

multiplexer 3. Multiplexer 3 converts the three color- mentioned, is the generation of digital synchronization 

descnptive signals to line-sequential format appearing codewords suppUed via channel 101 to assembler 10 to 

on a composite signal bus 34. The line-sequential com- 10 be coded and transmitted by transmitter 12 over the 

posite S!^ 13 applied to the input port of a pyramid data link for reception by the receiving apparatus of 

analyzer 4 m accordance with the mvention. The spa- another teleconferencing station. These synchroniza- 

tial-frequency analyses from pyramid analyzer 4 are tion codewords are also decoded within assembler 10 to 

suppbedm paraUeltoasamplefomatter5.F^^^ ^^^^^ its prioritization of codewords respectively 

rearrange the samples of the various spatid^^ 15 from buffer 9 and from digital audio coder 102. Anoth<i 

S^:^ti!JpuL^^^ aspect of this synchronizadon perfonned by ^^^^^^ 

DifferentSlpulTcLSmS^or^ S^o^«S^^^^ 

ftJL. - ...u- t. — „ 1 storage locations m FIFO date buffer 9 arc occu- 

of the adta-irame type, which compares samples seoa* j^^^i - ^ i ^ 

* J 1. J \» ^ w *6j*a Control circuitry 11 controls operation to keep 

rated by a frame time duration, but located sunilarlv 20 * j . '^^^k 

within raster space. Strings of successive zero™ ^If'^Lfit'f r^'' 'T°i 

the resulting differentki pulse code modulation ^.^^t^;^^.?^ ^'"V , 
(DPCM) signal are counted by a zero run-length en- ^ Asapart of its operation con^olcircmt 11 exchanges 
coder 68. A statistical encoder 8, which may bet Huff- ^f^ont^ retrace, vertical retrace and camera pjxel 
man encoder or a variant thereof, generates differing 25 f!^^ 'f^J^'' camera 2 via comiections 110, 
length self-parsing codewords responsive to each zero l^^ f information speci- 

run-length value and each non-zero value of the DPCM ^ analog-tOKhgital conversion circmtry m cam- 
signal, the length of the codeword depending on the ^ ^f^^ color-descnptive analog signals to gener- 
frequency of occurrence of the value being encoded. color-descnptive digital signals appUed to multi- 
The successive codewords are forwarded to a first- 30 P**^^ ^' Control circmtry 11 responds to the horizontal 
in/first-out (FIFO) data buffer 9. where they have time- retrace and vertical retrace information exchanged with 
division multiplexed with them synchronization and camera 2 to generate certain of the synchronization 
control codewords received from control circuitry 11 codewords it supplies to FIFO data buffer 9 via channel 
via a channel 101. FIFO data buffer 9 includes random- '^cse certain synchronizing codewords typically 
access memory (RAM) with respective addressable 35 "delude a beginning-of- frame codeword, an end-of- 
storage locations for each successive codeword re- frame codeword, and modulo-two frame count code- 
ceived, an accumulator for binary ones to generate words. Codewords for paning scan lines may also be 
write addresses for its RAM, an accumulator for binary generated. 

ones to generate read addresses for its RAM, and a Time-division multiplexer 3 is shown as comprising 
subtracter for calculating from the current ones of those 40 ^can line compressors 31, 32 and 33 respectively associ- 
addresscs how many of the storage locations in its RAM ^ted with separate ones of the color-descriptive digital 
are occupied, or filled. signals. Control circuitry 11 furnishes timing control 

The codewords read from data buffer 9 are supplied information to each of the scan line compressors 31, 32 
to an assembler 10, there to have time-interleaved with and 33 via respective multiple-conductor connections 
them synchronization codewords received via a chan- 45 l*^* and 115. 

nel 101 from control circuitry 11 and digital audio code- f^G- 2 shows a form that any one of the scan line 
words supplied from a digital audio coder 102. Assem- compressors 31-33 can take. Successive scan lines in 
bier 10 controls via a connection 103 its supply of code- camera 2 output signals are considered to be consecu- 
words from data buffer 9. Assembler 10 controls via a . lively iaumbered beginning with unity in each frame. A 
connection 104 its supply of codewords from digital 50 write-input multiplexer 120 responds to modulo-two 
audio coder 102. Assembler 10 reformats the resulting scan line count, as supplied from control circuitry 11, to 
succession of digital codewords into a serial stream of select odd-numbered scan lines of a camera 2 output 
bits for application to a code transmitter 12. Code trans- signal to the input/output bus of a first random-access 
mitter 12 transmits the code on a modulated carrier via memory 121 and to select even-numbered scan lines of 
a data link (not shown). E.g., the transmission may be a 55 that camera 2 output signal to the input/output bus of a 
phase shift keying (PSK) transmission of one of the second random-access memory 122. A preferable way 
known PSK types. to construct multiplexer 120 is using respective banks of 

One may view the FIFO data buffer 9 as performing tri-statc latches to supply data to input/output busses of 
a portion of an overall assembly process completed m RAMs 121 and 122. The rank of tri-state latches supply- 
assembler 10. One way to run the overall assembly 60 ing data to the input/output bus of RAM 121 is placed 
process is to divide digital audio into segments descrip- into a high-output-impedance state during even-num- 
tive of sound during respective camera 2 fields. A re- bered scan lines, so reading from RAM 121 is not inter- 
spective one of these segments is inserted each time fered with; and the rank of tri-state latches supplying 
after assembler 10 receives an end-of-fieid synchroniza- data to the input/output bus of RAM 122 is placed into 
tion codeword from FIFO data buffer 9. After the digi- 65 high-output-impedance state during odd-numbered scan 
tal audio segment data is again taken from FIFO data lines, so reading from RAM 122 is not interfered with, 
buffer 9, be ginning with a beginning-of-field synchrom- tsach of ihe RAx^s iii and 122 ima suluuicui ouuicas- 
zation codeword. What is important is that the overall able memory locations to store respective ones of each 
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of the samples in a scan line of camera 2 output signal generate cyclically the TDM strobe pulses for each of 

and is therefore denominated a "line store" RAM. the scan line compressors 31-33. 

Modulo-two line count is applied to the write enable Returning to FIG. 1, the responses of pyramid analy- 

connection of RAM 121, conditioning RAM 121 for zer 4 are supplied in parallel to converter 5 and are 

writing only during odd-numbered lines. Moduio-two 5 converted by fonnatter 5 to a single serial stream of data 

line count is supplied to a logic inverter 123, The result- words for application to differential pulse code modula- 

ing complement of line count supplied from inverter tor 6, which may be of a conventional design. It is pref- 

123 is applied to the write enable connection of RAM erable to use a DPCM where samples similarly located 

122, conditioning RAM 122 for writing only during in raster-scan space, but differenUy located in time, are 

even-numbered lines. 10 differentially combined since the thne-integrating prop- 

An AND gate 124 conditionally appUes modulo-two erties of the human visual system make quantization 

line count, during a unit-value TDM strobe pulse only, errors less noticeable. In modulator 6 the current-frame 

to the read enable connection of RAM 122 and to tri- samples of pyramid-analyzed video arc differentially 

state latch array 125. RAM 122 is thereby conditioned combined with the corresponding previous-frame sam- 

for reading during odd-numbered lines and tri-state P^^s in subtracter 61. The resulting difference signal is 

latch array 125 is thereby conditioned to connect the ^^^med to have at least one more bit per sample than 

mput/output bus of RAM 122 to composite signal bus the video output sigjials of c^era 2 m order to p^ 

34withlowsourceimpedancc,solongastheunit-value ^1^?^^.**? '^^^^^^ for<i<^ by the camera. In- 

TDM strobe pulse obtains. ^ difference signal may have a few additional 

An AND gate 126 conditionally applies the comple- during pyramid analysis in order to avoid 

mcntofmodulo-twolinecount,duringunit-valueTDM ^P^^t^de error givmg rise to apparent spattal phasmg 

strobe pulse only, to the read enable comiection of '^^f ^^^^^"^^"^ smooth edges). 

RAM 121 and to tri-state latch array 127. RAM 121 is J" « «>itsthe difference signals uito a num- 

thereby conditioned for reading durhig even-numbered „ ber of ge bms r«pectivety assig^^ 

r J * • * * 1 * I. i-vr* *L u J 25 tve ordmal numbers descnbable m appreciably fewer 

hnes and m-statc atch array 127 is thereby condbUoned ^. ^ ^ difference sigmds thcmsdvcs are. For ex- 

to apply the signal on the mput/output bus of RAM 122 ^pj^e to7^^ ^ iSed b^Tes^^Ive five-blt 

to composite signal bus 34vnth low source impedance, ^^^^ ^ ^^^^ ^^^^ ^ ^^^^ 

so long as the umt-value TDM stobe pulse obtains. ^^^^ ^ ^^^^^^ 

punng zero-value mtervals between .TDM fobc 3^ ^^cd to that bin, thereby to generate one ad- 
pulscs, neither of the tn-state latch arrays is conditioned ^^^^ ^ ^ ^^1^^^ ^^^^^ 
to provide low ^urcc mipedance to composite si^ ^ ^ ^^^^ predicted value of the current- 
bus 34. This m effect disconnects bus 34 from the HG. frame sample suppUed as subtrahend to subtracter 61. 
2 scan Une compressor, so bus 34 may have at least one ^he sum from adder 64 provides the predicted value for 
other scan hne compressor connected to it on a time- 35 ^^^^^ ^j^j^jj ^ delayed for a frame time by a 
interleaved basis. one-frame delay line 65. The one-frame delay line 65 is 

Modulo-two hne count also controls an address mul- typically provided by a RAM operated in a read-modi- 

tiplexer 128, which selects the output of a write address fied-write mode and addressed by a counter continu- 

generator 13 to the address port of RAM 121 during ^^jy clocked at the modulator 6 sample rate, 
odd-numbered lines and to the address port of RAM 40 Suppose quantizer 62 is a linear quantizer, one where 

122 during even-numbered lines. Address multiplexer quantization is carried out simply by rounding sub- 

128 also selects the output of a read address generator tractor 61 difference signal to the closest few-bit num- 

14 to the address port of the one of the RAMs 121 and jhen bin number decoder 63 can be replaced by 

122 not selected to receive the output of write address du-ect connection. Quantizer 62 may be one that has the 
generator 13. ... 45 boundaries of its range bins adjusted during operation 

Write address generator 13 typically includes a pixel responsive to a control signal received via connection 

counter reset to zero at the beginning of each scan line ^ from control circuitry 11. This adjustment is one of 

and counting throughout each scan line at. camera 2 those that may be made by control circuitry 11 respon- 

pixel scan rate for the particular image variable of con- give to information received via connection ^ that the 
ccm. This pixel counter supplies its count as write ad- 50 FIFO data buffer 9 is being under-filled or over-filled in 

dress output for generator 13. Read address generator unacceptable degree. 

14 typically includes a counter counting at higher rate In areas of the raster where there is not much frame- 

than the pixel counter in write address generator 13. to-frame change, the DPCM signal from modulator 6 

This higher counting rate provides for scan line com- will be zero-valued, presuming quantizer 62 assigns all 
pression. 55 low-valued input signals a range bin number of zero. In 

In practice, it is convenient to use a single pixel recognition of the fact that such lacli of frame-to-frame 

counter counting at Y scan rate to provide write ad- change is likely to occur a good percentage of the time, 

dresses to the line-store RAMs selected for writing in encoder 68 is used to encode the duration of a run of 

each of the scan line compressors 31-33. Where I and Q successive zero-valued DPCM samples with the objec- 
scan rates are octaves lower than Y scan rate, only the 60 tive of describing that run in fewer bits than the collec- 

more significant bits of the pixel count are used as write tive number of bits in these zero-valued samples. The 

addresses to the line-store RAMs in them selected for zero nm-length values may then be statistically encoded 

writing. together with non-zero values in the statistical coder 8. 

Also it is convenient to use a single higher rate Less frequently encoimtered zero run lengths may be 
counter to generate from the less signiuumii uiu of its CZ thztz efficiently coded ^^y ending a marker that such 

count read addresses for the line-store RAMs selected condition obtains to be followed by a hiatus in statistical 

for reading in each of the scan line compressors 31-33. coding during which time the run length value in 

The more significant bits of its count can be decoded to straight binary code is forwarded to data buffer 9 as 
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input signal. For a more complete discussion of these and to RAM 52 during even-numbered frames, and for 
techniques for furtiaer encoding the DPCM signal from applying read addresses from generator 54 to RAM 52 
modulator 6» one is referred to N. J. Feddc's U.S. pa- during odd-numbered frames and to RAM 51 during 
tent application Ser. No. 130,379 filed 8 Dec, 1987, even-numbered frames. A write multiplexer 56 re- 
entitled "MODIFIED STATISTICAL CODING OF 5 sponds to modulo-two frame count suppUed it via a 
DIGITAL SIGNALS" and assigned to General Heo- connection 57 from control circuitry H to direct 
trie Company. pyramid analyzer 4 results to RAM 51 for writing dur- 
Where zero-run-length encoding of DPCM signals ing odd-numbered frames and to RAM 52 for writing 
from modulator 6 is pursued, it is generally advanta- during even-numbered frames. A read multiplexer 58 
geous to have the formatter 5 pack the serial sample 10 responds to modulo-two frame count supplied it via a 
stream it supplies to modulator 6 such that zero run connection 59 finm control circuitry 11 to select read- 
lengths are longer in duration This desideratum can be out from RAM 52 to DPCM modulator 6 mput port 
more often achieved if one brings the pyramid analyses during odd^umbered frames and from RAM 51 to 
of the color variables arising from the same area of the DPCM modulator 6 input port during even-numbered 
scene presentation to camera 2 into better time align- 13 frames. . 

ment in the serial data stream supphed to modulator 6. One skilled in the art will recognize that variants of 
This is possible to do because pyramid transforms are formatter 5 where RAMs SI and 52 are only partial- 
localized spatial transforms. The differential delays of frame stores are possible as well. Furthermore, one 
pyramid components caused by line-sequential process- skilled in the art will appreciate that variants of format- 
ing should be reduced nisofer as possible. Also any time 20 ter 5 can be readily designed that win accommodate 
skew beitween various spatial frequency analyses intro- other types of pyramid analyzer 4 that do not use odd- 
duced by the iterative hierarchal filtering and decima- symmetry quadrature mirror filters. This, at some cost 
tion processes should be compensated for. in increased complexity. 

In pursuance of these goals formatter 5 includes two- FIG. 3 shows how the composite image field raster- 
frame store RAMs 51 and 52 for storing odd-fr^me 25 scanned from multiplexer 3 to the pyramid analyzer 4 
pyramid analyses and for storing even-frame pyramid can be considered in regard to the FIG. 1 video telecon- 
analyses, respectively. Preferably pyramid analyzer 4 is ferencing station as thusfar considered. A Y raster- 
one using successive ranks of quadrature mirror filters, scanned field 170, a Ci raster-scanned field 171 and a C2 
which avoids time skew and keeps the number of sam- raster-scanned field 172 arc arranged side by side. Scan 
pies in the pyramid analyses the same as the nimiber of 30 lines (not specifically shown extend across all these 
samples in the signal being analyzed. It is convenient to fields and line advance is considered to be in the vertical 
make the quadrature mirror filters separable, filtering direction (conventionally from top to bottom). If de- 
first in the direction parallel to line scan and then filter- sired, brief blanking intervals 173, 174 and 175 may 
ing in the direction perpendicular to line scan. Then, separate fields 170, 171 and 172 from each other to 
decimation in the .direction parallel to line scan reduces 35 reduce the possibility of cross-color among the Y, Ci 
the number of samples and the sample rate into the and C2 components in middle horizontal spatial-fre- 
ensuing transverse filtering. It is also convenient to use quency bands at left and right extremes of the display, 
poly-phase filtering in the direction parallel to line scan Each line of the Y raster and the ensuing blanking inter- 
to halve sample rate in that direction. If odd-symmetry vaJ, if any, occupies two-thirds of a camera 2 scan line 
kernels are employed in the quadrature mirror filters, a 40 time (including trace and retrace intervals). Each line of 
one-pixel offset occurs between the decimated high- the Ci raster and the ensuing blanking interval, if any, 
pass and low-pass spatial filter responses in each direc- occupies one-sixth of a camera 2 scan line. So does each 
tion. Accordingly, the pipelined pyramid analyzer 4 line of the C2 raster and the ensuing blanking interval, 
results will be in time-division-multiplexed relationship, The 4:1 horizontal compression of C| and signals 
which simplifies timing the writing and reading of 45 respective to Y signal is accomplished in the scan line 
RAMs 51 and 52. compressors 31-33 and is permissible since camera 2 

A write address generator 53 and a read address gen- low-pass filters Ci and C2 to narrower bandwidth than 
erator 54 in formatter 5 can then generate respective Y in the horizontal direction This 4:1 horizontal corn- 
sequences of addresses at the same clock rate as each pression of Ci and C2 is in effect a frequency-quadru- 
other and as the pixel clock rate of the pyramid analyzer 50 pling process that multiplies the spatial frequencies of 
4. Control circuitry 11 supplies this same rate clock to Ci and C2 to extend over a bandwidth commensurate 
analyzer 4 via a multiple-conductor connection 116. to with that of Y. Highcr-horizontal-frcquency compo- 
write address generator S3 via a multiple-conductor nents of the analysis by pyramid analyzer 4 will no 
connection 117 and to read address generator 54 via a longer be zero-valued as the Cj and C2 rasters are 
multiple-conductor connection 118. These multiple- 55 scanned. This means that the pyramid analysis of Ci and 
conductor connections 116, 117 and 118 also carry line C2 proceeds to lower octaves than the pyramid andysis 
synchronization and frame-count-modulo-two informa- of Y. Another effect is that when higher spatial-fre-. 
tion from control circuitry 11 to analyzer 4, to write quency components of the FIG. 3 image field cannot be 
address generator 53, and to read address generator 54, transmitted owing to FIFO data buffer 9 tending to 
respectively. (The relationship between the write ad- 60 overfill, the viewing distance at which the loss horizon- 
dresses generator 53 and the read addresses supplied by tal resolution is first perceptible mcreases similarly for 
read address generator 54 may also incorporate some chrominance and for luminance. This occurs automati- 
degrec of positional coding to provide a better degree of cally without having to treat the C \ and C2 components 
security for the video teleconferencing transmissions.) of the composite image field separately from the Y 

An address multiplexer 55 responds to modulo-two 65 component 

irame-couni suppllcu ii vm a cOuucction 119 frcm cos- The FIG. 3 coTiiposite iis?.*'? t?'^*^^ Veens m Inw 

trol circuitry 11, for applying write addresses from aspossible the rate of read-out from the one of the scan 

generator 53 to RAM 51 during odd-numbered frames line compressors 31-33 temporarily storing luminance 
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signal. This, saves -operating power for the scan line 
compressor. (While it is desirable to -decrease the num- 
ber of samples per scan line to just above Nyquist limit 
insofar as saving power in the multiplexer 3, pyramid 
analyzer 4» formatter 5 and differential pulse code mod- S 
ulator 6 is concerned, such minimal sampling rate exac- 
erbates the problem of amplitude quantization in quan- 
tizer d2 generating alias spatial frequencies. Better de- 
sign practice is to choose a sample rate at least twice 
Nyquist rate to keep aliasing acceptably low, as known 10 
to the present inventor from a co-worker» £. H. Adel- 
son. Some of the increase in the data rate from modula- 
tor 6 is offset by the fact of zero run lengths tending to 
be longer.) Where there is less concern for saving 
power in the scan line compressors 31-33 and formaner 
S, one may contemplate alternative modes of operation 
in accordance with other aspects of the invention. 

FIO. 4 shows another way the composite image field 
raster-scanned from multiplexer 3 to the pyramid analy- 
zer 4 can be considered. In this mode of operation each 
of the scan line compressors 31-33 of FIG. 1 is read out 
in a respective period extending over substantially one- 
third a scan line of camera 2. That is, for example, the Y 
(or G) signal raster 170' is located in the left third of the 
composite image field the Ci (or R) raster 171' is located 
in the middle third of the composite image field; and the 
C2 (or B) raster 172' is located in the right third of the 
composite image field. Component rasters 170-172' 
may be separated by short blanking intervals 173', 174' 
and 175' to reduce or avoid the possibility of cross-color 
in middle spatial frequencies. 

Where scan line compressors 31, 32 and 33 provide 
temporary storage for Y, Ci and C2 signals respectively 
the upper spatiat frequencies of Ci and Ci in both hori- 35 
zontal and vertical directions are less necessary than 
those of Y in satisfying the spatial resolution needs of 
the human visual system. Accordingly, the components 
of the spatial-frequency analyses of Ci and C2 descrip- 
tive of these higher spatial frequencies (to the extent 40 
camera 2 filtering permits them to exist) may be dis- 
carded in the pipeline output from the formatter 5. No 
provision is provided in RAMs 51 and 52 for storage of 
these components. Write multiplexer 56 protocol is to 
ignore these high-spatial-frequency components of Ci 45 
and C2 supplied to its input port and not write them into 
memory. (This may be done by withholding write en- 
able signals firom RAMs 51 and 52 at appropriate times 
if these RAMs are of a type permitting. Read multi- 
plexer 58. protocol is to select zero for application to 50 
differential pulse code modulator 6 during the times the 
discarded samples would appear, rather than reading 
out stored samples from a selected one of RAMs 51 and 
52. (If the RAMs 51 and 52 arc of a type permitting, this 
may be done by withholding read enable signals from 55 
botii of them at appropriate times.) 

At the same time, samples being read from the one- 
frame delay line €5 are replaced by zeros. This causes 
the DPCM signal from modulator 6 to be zero-valued 
during the times Ci and C2 high-spatial-frequence com- 60 
ponents are being discarded Also, however, since zeros 
are circulated back to adder 64 rather than the samples 
read from the one-frame delay line 65, one avoids these 
samples causing mistaken predictions of input signal in 
subsequent frames. The long zero runs invariably ap- 63 
pearing at the times the omitted C\ and C2 components 
would occur in the pipeline output from formatter 5 are 
efficiently coded by the zero run length encoder 68. 



Where scan line compressors 31, 32 and 33 provide 
temporary storage for green, red and blue signals re- 
spectively, one may arrange for high-spatial-frequency 
green to substitute as a mixed highs signal and then 
discard the high-spatial-fi-equency components of red 
and blue. (This expediency is sometimes used in the 
horizontal direction in color television broadcasting.) If 
this expedient is pursued, appropriate arrangements 
must be made in the color matrixing circuitry of the 
receiving apparatus to add in the high-spatial-frequency 
green components with the low-spatial-frequency com- 
ponents of each of the green, red and blue signals. 

Returning consideration to the operation of the FIG. 
1 sending apparatus for transmittmg the composite 
image field of FIG. 3, where pyramid analyzer 4 is of 
the quadrature mirror filter type, samples of the verti- 
cal-direction high-spatial-firequency components of the 
Ci and C2 signals are prefen^ly replaced by zeroes in 
the pipelhie output of formatter 5 using appropriate 
write multiplexer 56 and read multiplexer 58 protocols. 
This is done in accordance with the teachings set forth 
above in regard to operation of the FIG. 1 sending 
apparatus for transmitting the composite image field of 
FIG. 4. 

FIG. 5 shows a receiving apparatus for a telecon- 
ferencing station, which receiving station is suitable for 
receiving signal from a FIG. 1 sending station. A code 
receiver 18 converts signal received from a data link 
(which by way of example may be phase shift keyed 
signal) to a succession of binary code values parsed into 
codewords by a parser 19. Certain of the codewords 
from parser 19 are synchronization and control code- 
words decoded by control circuitry 21 for the FIGURE 
S receiving apparatus. Others of the codewords are 
digital audio codewords decoded by a digital audio 
decoder 191 to recover digital audio signal converted to 
analog audio signal by a digital-to-analog converter 192. 
The analog audio si^ial is amplified by an amplifier 193 
for application to a loudspeaker 1SM-. 

Of particular interest to the invention are the statisti- 
cal codewords descriptive of video. These statistical 
code words are decoded (e.g. by look-up table read- 
only memory) in a statistical decoder 20. Where the 
statistical encoder 8 of the FIG. 1 sending apparatus is 
of a type where statistical coding procedure is changed 
from time to time depending on the occupancy of stor- 
age locations of the FIFO data buffer 9, statistical de- 
coder 20 will receive instructions from control circuitry 
21 to accommodate its decoding scheme to the changed 
coding scheme, these instructions being communicated 
via a connection 22 from control- circuitry 21 to statisti- 
cal decoder 20. (Control circuitry 21 generates these 
instructions responsive to certain synchronization and 
control words appearing in parser 19 output signal.) 
Statistical decoder 20 directly specifies the non-zero 
sample values that are the nominal values of the range 
bins in the FIG. 1 digital pulse code modulator 6, avoid- 
ing the need for separate bin number decoder in a differ- 
ential pulse code demodulator 25 used later on in the 
reconstitution of the color images. Statistical coder 20 
also specifies in code terms the length of zero runs that 
occur from time to time among non-zero samples. The 
successive digital output signal values from the statisti- 
cal decoder 20 are supplied as input signal to a first- 
in/first-out data buffer 23. 

FIFO data butter Z5 clocks oui non-zero samples ai 
pyramid-analyzer 4 pixel scan rate responsive to clock- 
ing signals supplied to buffer 23 from control circuitry 
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21, These non-zero samples pass tlirough a run length selected for writing in each successive frame. Read 

code unpacker 24 to differential pulse code demodula- multiplexer 78 responds to control signals supplied from 

tor 25. Run length code unpacker 24 responds to a zero control circuitry 21 via a connection 79 to read, from 

run length code appearing at data buffer 23 output port the one of RAMs 71 and 72 selected for reading in each 

to select to demodulator 25 a zero value rather than the 5 successive frame, to the mput port of pyramid synthe- 

output value from buffer 23. Unpacker 24 further re- sizer 26 appropriate to the current sample of input signal 

sponds to the zero run length code to enable a counter thereto. 

therewithin. This counter counts pixel scan rate pulses Pyrainid synthesizer 26 regenerates the composite 

supphed unp^er 24 firam control circuitry 21 via a field image samples, with three color variables appear- 

connection 211 Unpacker 24> under control of this 10 mg in line-sequential order. This serial stream of sam- 

counter i^iternal to itsel^jreapphes the value to pies, similar to that originally provided on the FIG. 1 

demodulator 25 ^e number of times specified by the composite signal bus 34rirsuppHed by pyramid synthe- 

run length code. Unpacker 24. abo under control of its ^ composite signal bus 27 m theFIG. 

mt^ counter, sends back to FIFO data buffer 23 a 2 sending station. Scan l^deco^^sors 41. 42 and 43 

Mtsigna^ddaymgthen^dockoutfro^ ,5 perform respective time^iivision d^nnultiplering o^^^ 

23 untd the zero run has been regenerated in its entirety ^Z.^ r.^ i u "*^"J."P**=^o 

for appUcation to demodulator 25. ^P^^ ^ *° 

. D^^ential pulse code d^cSlator 2^ £™oSU^^^^ 

adder 251 clocked at pyramid analyzer 4 pixel scan rate. ^^I'^M^^^^'Tr^ for apphcation to color matru- 

Adder 251 has a firsi addend input port to which the 20 ^^T^^ fr^^-. fT^^ '^^'^'^Jt^T^^'^r 

output port of run length code 24 comiects. * "^uf^mx*^?* ^? ^ 

has a s^ond addend i^ut port,Zi has a sum ou^t ^ digital vxdep signal. These 

port for supplying the digital pulse code demodulator f 'fj! ^^^^ ^ «^PP^*^ ^ ^ video mom- 

25 output signal. This signal is delayed by one frame Display synchromzation signals are supphed to 

time in a clocked delay line 252 for application to the 25 "^^J^ momtor 29 from control circuitry 21 via a bus 

second addend input port of adder 251, thereby com- ^ ._. . , , 

pleting the demodulator loop. Differential pulse code Consider more particularly now an example of how 

demodulator 25 output signal resembles the signal that ^® decompressors 41-43 can work. Responsive to 

formatter 5 suppUes to digital pulse code modulator 6 in signals suppHed it from control circuitry 21 via a 

the FIG, 1 sending apparatus. 30 con^iec^o^ 216 scan line decompressor 41 selects timc- 

The parallel samples to serial samples conversion and compressed lines of Y signal samples appearing on bus 

any other permutation procedure carried out by format- ^ stretches them to fit the trace time of video 

ter 5 m the FIG. 1 sending apparatus has to be undone monitor 29. Responsive to timing signals supplied it 

in the FIG. 5 receiving apparatus to obtain input signals control circuitry 21 via a connection 217 scan line 

for a pyramid synthesizer 26. A sample formatter 7 in 33 decompressor 42 selects time-compressed lines of Ci 

the FIG. 2 receiving station does this, supplying to ^S^^ samples appearing on bus 27 and stretches them 

pyramid synthesizer 26 signals generally similar to those *P ^* ^® ^® ^^co monitor 29. Responsive to 

supplied from the pyramid analyzer 4 in FIG. 1. timing signals supplied it from control circuitry 21 via a 

Formatter 7 includes an odd-frame pyramid frame connection 218 scan line decompressor 43 selects time- 
store RAM 71 and an even-frame pyramid frame store 40 compressed lines of Cz signal samples appearing on bus 
RAM 72 analogous to RAMs 51 and 52, respectively, of ^ stretches them to fit the trace time of video 
FIG. 1 formatter 5. Formatter 7 also includes a write monitor 29, 

address generator 73, a read address generator 74, an ^ shows in greater particularity a structure for a 

address multiplexer 75, a write multiplexer 76 and a scan line decompressor such as 41, 42 or 43, Successive 

read multiplexer 78. 42 scan lines in video monitor 29 are considered to be 

Write address generator 73 responds to synchroniz- consecutively numbered beginning with unity in each 
ing information, which is supplied via a connection 212 frame. A write-input multiplexer 150 responds to modu- 
from control circuitry 21, to scan the storage locations lo-two scan line count, as supplied from control dr- 
in the one of RAMs 71 and 72 selected for writing in the cuitry 21, to select odd-numbered scan lines of compos- 
current frame This scan is in the same pattern that read SO ite signal from bus 27 to the mput/output bus of a first 
address generator 54 scans in the FIG. 1 sending appa- random-access memory 151 and to select even-iium- 
ratus formatter 5. Read address generator 74 responds bered scan lines of composite signal to the input/output 
to synchronizing information, which is supplied via a bus of a second random-access memory 152. A prefera- 
connection 213 from control circuitry 21, to scan the ble way to construct multiplexer 150 is using respective 
storage locations in the one of RAMs 71 and 72 selected 55 fanks of tri-state latches to supply data to the input/out- 
for reading in the current frame. This scan is in the same put busses of RAMs 151 and 152, for reasons discussed 
pattern that write address generator 53 scans in the indetailaboveinregardtomultiplexer 120. Eachof the 
HO. 1 fonnatter 5. RAMs 151 and 152 has sufficient addressable memory 

Address multiplexer 75 receives via connection 214 locations to serve as a line store, 

from control circuitry 21 an indication of whether an 60 Modulo-two hne count is applied to the read enable 

odd frame is to be written and an even frame is to be connection of RAM 151, conditioning RAM 151 for 

read or vice versa. Multiplexer 75 responds to apply the reading only during odd-numbered lines. Modulo-two 

write address from generator 73 and the read address line count is supplied to a logic inverter 153. The result- 

from generator 74 to the appropriate ones of RAMS 71 ing complement of line count supplied from inverter 

and 72 for eacft successive frame 'rr'tlU iu'i;]tiplc;:cr 75 £5 15? is appU**^ the read enable connection of RAM 

responds to control signals supplied from control dr- 152. Modulo-two count also controls a read-output 

cuitry 21 via a connection 77 to write the output signal multiplexer 155 which selects as input signal to an input 

from demodulator 25 to the one of RAMs 71 and 72 port of color matrixing circuitry 28 the read-out from 
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the one of RAMs 151 and 152 conditioned for reading codeword indicatmg the changes in bin allocation and 
by receiving unit-value read enable signal. in coding table selection^ which codeword is transmit- 

An AND gate 154 conditionally applies modulo-two ted via connection 101 to the FIFO data buffer 9. In the 
line count, during unit-value TDM strobe pulse only, to FIG. 5 receiving apparatus control circuitry 21 detects 
the write enable connection of RAM 152. RAM 152 is 2 this codeword and responds to its detection by sending 
thereby conditioned for writing during the portions of instructions (via connection 22) to statistical decoder 
odd-numbered lines when unit-value TDM strobe 20. Statistical coder 20 responds to these instructions to 
pulses occur. modify its decoding table to suit the binning in quantizer 

An AND gate 15^ conditionally applies the comple- 62 and the coding table of statistical encoder 8. This 
ment of modulo-two line count, during unit- value TDM 10 modification can be made by selecting a difTerent de- 
strobe pulse only, to the write enable connection of coding table from a group of look-up tables stored in 
RAM 152. RAM 152 is thereby conditioned for writing read-only, memory. 

during the portions of even-numbered lines when unit- Alternatively, if in the FIG. 1 sending apparatus the 
value TDM strobe pulses occur. data buffer 9 tends to overfill, control circuitry 11 can 

Modulo-two line count also controls an address mul* 15 respond by modifying the operation of read multiplexer 
tiplexer 158, which selects the output of a write address 58 so it selects zeros during time slots normally filled by 
generator 16 to the address port of RAM 151 during the samples of the discarded high-spatial-frequency 
odd-numbered lines and to the address port of RAM spectra. At the same time, samples read from the one- 
152 during even-numbered lines. Address multiplexer frame delay line 65 arc replaced by zeros. The differen- 
158 also selects the output of a read address generator 20 tial pulse code modulator 6 then supplies zero bin num- 
17 to the address port of the RAMs 151 and 152 not bcrs diuring these time slots. This reduces the number of 
selected to receive the output of write address genera- non-zero values to be coded by the zero-run length 
tor 16. encoder 68 and statistical coder 8 in the FIG. 1 appara- 

Read address generator 17 typically includes a pixel tus, and replaces them with long zero runs that can be 
counter reset to zero at the beginning of each scan line 25 efficiently coded. This tends to reduce the fill rate of 
and counting throughout each scan line at video moni- FIFO data buffer 9 in a pronounced way. 
tor 29 pixel scan rate for the particular image variable of It is noted in passing that a convenient way to con- 
concern. This pixel counter (not explicitly shown in struct the write and read address generators of a send- 
FIG. 6) supplies its count as read address output for ing or receiving apparatus around a read-only memory 
generator 17. Write address generator 16 typically in- 30 storing the successions of write and read addresses in 
eludes a counter counting at higher rate than the pixel • alternate phases in sequentially addressed storage loca- 
coimter in read address generator 17. This higher count- tions of die read-only memory. The addresses of this 
ing rate provides for scan line de-compression. read-only memory receive their most significant bits as 

In practice, it is convenient to use a single pixel binary-coded instructions, from the control circuitry of 
counter counting at Y scan rate to provide read ad- 33 the sending or receiving apparatus, and the less signifi* 
dresses to the line-store RAMs selected for writing in cant bits of the addresses for this read-only memory are 
each of the scan line de-compressors 41-43. Where I generated in sequence by a counter clocked at the sam- 
and Q scan rates are octaves lower than Y scan rate, pie rate through the formatter, 
only the more significant bits of the pixel count are used If one of the write address and read address sequences 
as read addresses to the line-store RAMs in them se- 40 consists of consecutive numbers, an alternative con- 
lected for writing. struction that conserves read-only memory can be cm- 

Also it is convenient to use a single higher-rate ployed. A counter generates the address sequence con- 
counter to generate from the less significant bits of its sisting of consecutive numbers, which address sequence 
count write addresses for the line-store RAMs selected addresses a read-only memory storing the other address 
for reading in each of the scan line decompressors 45 sequence. The read and write addresses are strobed 
41-43. The more significant bits of its count can be alternately, one from the counter output and the other 
decoded to generate cyclically TDM strobe pulses for from the read-only memory output Read-only memory 
each of the scan line decompressors 41-43. is halved in siz^ so is its rate of reading. 

A more coherent description will now be made of the One armed with the foregoing disclosure and skilled 
steps the control circuitry 11 of the FIG. 1 sending 50 in the art of designing systems for the digital transmis- 
apparatus takes to control the occupancy of the FIG. 1 sion of video and accompanying sound will be equipped 
FIFO data buffer 9. This description will also refer to to design many variants of the apparatuses described 
the steps taken in concert therewith by the control herein. This should be borne in mind in construing the 
circuitry 21 of the FIG. 5 receiving apparatus for ad- broader ones of the claims which follow this applica- 
justing the operation of the receiving apparatus to track 55 tion. Some illustrative examples of design refinements 
the adjusted operation of the FIG. 1 sending apparatus. follow. 

In the FIG. 1 sending apparatus control circuitry 11 Poly-phase operation of pyramid analyzer 4 to reduce 
senses the occupancy of the storage locations in data the sample rates therein has been disclosed above in 
buffer 9 via connection 91 therefrom. regard to FIG. 1 sending apparatus* Poly-phase opera- 

If the data buffer 9 tends to overfill, control circuitry 60 tion can be initiated before pyramid analyzer 4 and scan 
11 can send control signals via connection 66 to adjust line compressors 31-33 to decrease the sample rate 
the binning done by quantizer 62 to increase the width through the line compressor as well as in the pyramid 
of the zero bin, or to more coarsely quantize, or to da analyzer 4. This procedure allows the use of slower 
both. If such steps are taken, signals can be sent to statis- digital hardware and some saving of operating power, 
tir^i encoder 8 to modify its coding table to suit-^.g. 65 In the FIG. 5 receiving apparatus, sinoilar advantages 
by selecting a different coding table from a group of obtaizi -^ith poly-phase opernHnti in the direction paral- 
look-up tables stored in read-only memory. Control lei to scan lines both in the pyramid synthesizer 26 and 
circuitry 11 will then also generate a synchronizing in the scan line decompressors 41-43 as well. 
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The need for scan line compression of Y in an early simple time-division multiplexer as well. The cost of 
stage of the sending apparatus of a video teleconference doing this is that zero run lengths are not as likely to be 
ing station can be avoided by pyramid analysis of the Y long, encryption of video cannot be done in formatter 5, 
signal separately from pyramid analysis of the Ci and and decryption of video cannot be done in formatter 7. 
C2 signals* which are placed into line-sequential format 5 While the invention has been specifically described in 
without including Y signal for pyramid analysis in par- connection with the color transmission of telecon- 
alld with Y signal The two pyramid analyzers, one for ferencing video data, the invention is applicable to 
Y signal and the other for line-sequential Ci and C2 other image processing problems. The line-sequential 
signals, are provided with respective, formatters. These time-division-multiplexing of other independent- varia- 
fonnatters have read multiplexing that time interleaves 10 ble image data such as stereoscopically related image 
their respective output signals to the input port of a data in connection with pyramid processing is a useful 
single differential pulse code modulator, and the re- step in the digital transmission of the image data* for 
mainder of the sending station is per FIG. 1. In addition example. The Adelson and Burt inventions mentioned 
to saving a scan line compressor for Y signal, scan line in the first paragraph of the BACKGROUND OF THE 
compressors for the Ci and C2 signals may be operated 15 INVENTION can also be modified to usefully employ 
at lower output rates. The pyramid analyzers are opera- the present invention, particularly in the lower-spatial- 
ble at lower rates, although two are required. The rates frequency processing where sample rates are lower, 
of writing random access memory in the formatters is The line^quential time-division multiplexing of video 
reduced. This alternative makes tt possible to achieve signals derived from cameras viewing entirely different 
high-resolution color transmission at reduced memory 20 scenes can be used in connection with pyramid process- 
speeds. Scan line compression for Y takes place in the ing in still other embodiments of the present invention, 
formatter. Analogously, the need for a scan line decom- Embodiments of the present invention may use inter- 
pressor for Y in the final stages of the receiving appara- laced raster-scanned video or may use progressive scan, 
tus of a teleconferencing station can be avoided, by as one skilled in the art will readily appreciate, 
using its formatter for decompressing Y and using sepa- 25 What is claimed is; 

rate pyramid synthesizers for Y luminance signal and 1. Apparatus for performing two-dimensional s{)atial- 
for the time-division-multtplexer Ci and C2 chromi- frequency filteringsof each of a plurality of signals each 
nance signals. Variants of this arrangement wherein descriptive of a raster-scanned image field sampled-data 
high-spatial-frequency luminance pyramid processing is variable, said raster-scanned hnage field sampled-data 
done in parallel with the pyramid processing of line- 30 variables having the same number of scan lines in their 
sequenti^y time-division multiplexed low-spatial-fre- respective raster scans, said apparatus comprising: 
quency luminance and chrominance components are a respective time compressor for each of said plural- 
attractive designs. This procedure is possible because of ity of signals, having the capability of time com- 
the general generating precepts discussed in the next pressing a full scan line of its respective signal; 
paragraph. 35 means for timing the operation of said time compres- 
Each stage of two-dimensional pyramid analysis cus- sors to time-division-multiplex said signals for in- 
tomarily involves 2:1 decimation in the direction paral- terleaving successive ones of their respective scan 
lei to line scan and 2:1 decimation in the direction per- lines on a cyclic basis; thereby to form a line- 
pcndicuiar to line scan. The latter decimation involves sequential composite signal which is a raster- 
the discarding of alternate scan lines of filter response in 40 scanned sampled-data description of a plurality of 
the signal for application to the next stage of pyramid side-by-side image fields; and 
analysis. This is a species of line compressor operation, a first two-dimensional spatial-frequency filter recep- 
it may be noted. Accordingly, if two raster-scanned tive of said composite signaL 
video signals having their respective line scans synchro- 2. Apparatus as set forth in claim 1 wherein said rast- 
nized and having a similar number of samples per scan 45 er-scanned image field sampled-data variables are raster 
line are subjected to their respective first stages of scanned parallel in time. 

pyramid analysis in parallel, and if the 2:1 decimation of 3. Apparatus as set forth in claim 1 wherein at least 
their pyramid responses are in different rather than the two of said raster-scanned image field sampled-data 
same spatial phasing, a multiplexer operated at line scan variables have the same number of samples in each of 
rate can combine the signals for application to the next 50 their respective scan lines. 

stage of pyramid analysis in a line-sequential manner. 4. Apparatus as set forth in daun 1 wherein said first 
This allows the later stages of pyramid analysis for these two-dimensional spatial-frequency filter is one of a plu- 
signals to be performed on a time-division multiplexed rality of two-dimensional spatial-frequency filters ar- 
basis in the same digital hardware, in accordance with ranged as a pyramid analyzer for analyzing said raster- 
the present invention. Pyramid synthesis of two signals 55 scanned sampled-data description of a plurality of side- 
using shared digital hardware in all but the tinal stage is by-side image fields into component spatial-frequency 
also possible in accordance with the present mvention. spectra in sampled-data form. 

The desirability of pyramid processing using separa- 5. Apparatus as set forth in claim 4 in combination 
ble quadrature mirror filters with odd-symmetry ker- with: 

nels has been pointed out, m regard to the time-inter- 60 a formatter for arranging the samples of said compo- 
leaving of the spectral components when writing into nent spatial-frequency spectra into a serial stream 

formatter 5 of the FIG. 1 sending apparatus and when of samples. 

reading out from formatter 7 of * the FIG. 2 receiving 6. A combination as set forth in claim 5 including: 
apparatus. Where formatter 5 is simply called upon for a differential pulse code modulator for generating a 
converting parallel streams of data into a serial stream 65 differential pulse code response to said serial 
VI uaia, ilio- ^mi uuuc wiiliuui iiccu lui iciapurury sUcam of samples. 

Storage, formatter 5 degenerating to a simple time-divi- 7. A combination as set forth m claim 6 wherein said 
sion multiplexer. Formatter 7 can in such case be just a differential pulse code modulator is of a type differen- 
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dally comparing samples separated by a frame time 
delay. 

8. A combination as set forth in claim 6 further in- 
cluding: 

means for statistically encoding said differential pulse S 
code to generate a succession of self-parsing statis- 
tical codes. 

9. A combination as set forth in claim 8 wherein said 
means for statistically encoding said differential pulse 
code includes: 10 

a zero run length encoder for determining the lengths 
of runs of zeroes in said differential pulse code, and 

means for selecting the statistical codes for zero-value 
samples in said differential pulse code dependent 
upon the length of zero runs in which those sam- 15 
pies are included. 

10. A sending apparatus, comprising in addition to the 
combination of claim 8: 

means for generating digital audio codes; 

means for generating synchronizing codes indicative 20 
of the framing of said raster-scanned sampled-data 
description of a plurality of side-by-side image 
fields and of the signals derived therefrom; 

means for generating a serial stream of bits from said 
statistical codes, said digital audio codes, and said 25 
synchronizing codes; and 

a transmitter for transmitting over a data link a signal 
responsive to said serial stream of bits. 

11. A combination of: 

a pyramid synthesizer, receptive of the component 30 
spatial-frequencyspectra in sampled-data form of a 
raster-scanned sampled-data description of a plu- 
rality of side-by-side image ^dds, for synthesizing 
said raster-scanned sampled-data description of a 
plurality of side-by-side image fields; and 35 

a respective time decompressor for each of said plu- 
rality of sidc-by-side image fields, arranged to ac- 
cept as its respective input signal samples descrip- 
tive of its respective image field and to provide a 
raster-scanned output of its respective image field 40 
synchronously with each other of said time decom- 
pressors. 

12. A receiving apparatus, comprising in addition to 
the combination of claim 11: 

a receiver for receiving from a data link a signal, 45 
responsive to which said receiver supplies a serial 
stream of bits; 

means responsive to said serial stream of bits for sepa- 
rating digital audio codes, synchronizing codes 
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indicative of the framing of said raster-scanned 
sampled-data description of a plurality of side-by- 
side image fields, and statistical codes descriptive 
of a differential pulse code modulation signal en- 
coding said raster-scanned sampled-data descrip- 
tion of a plurality of side-by-side image fields; 

means for decoding said statistical codes to recover 
said differential pulse code modulation signal; 

a demodulator for said differential pulse code modu- 
lation signal; 

a formatter for arranging data samples from said de- 
modulator to supply to said pyramid synthesizer 
said component spatial-frequency spectra in sam- 
pled-data form of the raster-scanned description of 
a plurality of side-by-side image fields; 

a video monitor for providing a display; 

means for applying video input signals to said video 
monitor in response to the raster-scanned outputs 
of their respective image fields supplied by said 
time decompressors; and 

means responsive to at least some of said synchroniz- 
ing codes for supplying said video monitor infor- 
mation with which to synchronize its display. 

13. A method for analyzing the spatial-frequency 
spectra of first and second raster-scanned video signals, 
said method comprising the steps of: 

time-division multiplexing said first and second rast- 
er-scanned video signals together to form a line- 
sequential third raster-scanned video signal, and 

performing a pyramid analysis of said third raster- 
scanned video signal. 

14. A method for synthesizing fu^t and second raster- 
scanned video signals from the spatial-frequency spec- 
tra of a third raster-scanned video signal of a type as is 
formed by line-sequential time-division multiplexing of 
said first and second raster-scanned video signals, said 
method comprising the steps of: 

performing a pyramid synthesis of said third raster- 
scanned video signal f^om its said spatial frequency 
spectra, 

time-division demultiplexing said third video signal 
for separating samples descriptive of scan lines of 
first video signal and samples descriptive of scan 
lines of second video signal, and 

resampling each set of separated samples so the raster 
scanning of said first video signal and the raster 
scanning of said second video signal are in synchro- 
nism with each other. 

***** 
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